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1.  Introduction 


Unmanned  ground  and  air  vehicles  (UGVs  and  UAVs)  embody  a  critical  capability  of  future 
U.S.  Army  systems  and  will  be  used  for  force  projection  and  as  force  multipliers.  They  are 
expected  to  operate  in  a  semiautonomous  fashion,  in  which  an  operator  will  lay  out  a  mission 
plan  for  the  vehicle  to  execute  with  little  or  no  operator  interaction;  the  operator’s  function  will 
be  to  monitor  the  progress  of  the  unmanned  systems.  If  the  situational  demands  exceed  the 
capabilities  of  the  vehicle,  the  operator  will  need  to  intervene  to  complete  the  mission  by  either 
changing  the  mission  plan  or  by  assuming  direct  control  of  the  vehicle  by  teleoperation  until  the 
vehicle  can  resume  semiautonomous  operation. 

Effective  operator  control  interfaces  for  UAVs  and  UGVs  are  essential  in  both  the  mission 
planning  phase  and  any  intervention  phase.  A  complicating  element  of  operator  control  is  that  it 
is  not  always  performed  while  the  vehicle  is  still  or  idle;  it  may  occur  while  the  operator  is  in  a 
moving  vehicle  on  cross-country  or  relatively  undeveloped  (e.g.,  gravel  road)  terrain. 
Throughout  years  of  U.S.  Army  Research  Laboratory  (ARL)  workstation  development  for 
demonstrations  and  experiments  based  on  operations  on  the  move  (OTM),  it  has  been 
demonstrated  that  operation  of  conventional  computer  control  devices  in  a  moving  vehicle  is 
challenging  at  best.  Vehicle  roll,  pitch,  yaw,  and  vibration  during  travel  can  make  common 
cursor  controllers  like  the  mouse  difficult  to  impossible  to  use.  In  fact,  the  mouse  was  so 
difficult  to  operate  OTM  in  this  vehicle  that  it  was  immediately  discarded  and  replaced  with  a 
trackball  during  the  demonstration  and  development  phases  of  outfitting  the  vehicle.  One 
example  is  the  use  of  a  handheld  trackball  during  the  Office  of  the  Secretary  of  Defense 
Robotics  DEMO  II  from  1992  to  1996.  Human-robotic  interface  (HRI)  devices,  such  as  those 
needed  for  control  of  unmanned  systems,  must  be  researched  in  these  challenging  environments 
to  optimize  operator  performance. 

This  study  was  run  to  determine  the  usability  of  control  devices  for  the  in-vehicle  control  of 
unmanned  robotic  systems.  Control  devices  commonly  known  as  console  controllers  or  game 
pads  have  proven  successful  for  use  in  commercial  computer  video  games.  The  goal  of  this 
study  was  to  benefit  the  Army  by  gathering  and  using  Soldier  performance  data  for  the  use  of 
common  computer  controllers,  (i.e.,  joystick  and  trackball)  and  the  console  controller,  to 
establish  performance  baselines  for  computer  control  devices  in  Army  vehicles  moving  on 
various  terrain  conditions.  A  second  goal  was  to  provide  information  that  may  allow  the  Army 
to  leverage  the  large  controller  design  investment  already  made  by  the  world- wide  gaming 
industry  to  improve  the  Soldier/controller  interface  in  Army  control  systems  and  provide  design 
guidelines  for  controllers  in  Army  OTM  applications. 
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As  a  part  of  the  Distributed  Aperture  System  Army  Technology  Objective  effort,  a  literature 
search  for  information  regarding  the  usability  of  console  game  controllers  revealed  virtually  no 
research  regarding  the  use  of  controller  devices  in  moving  environments  such  as  military 
vehicles.  Limited  examples  were  found,  such  as  an  evaluation  of  a  handheld  keyboard  control 
(Ting  and  Hedge,  2001)  that  evaluated  a  unique,  handheld,  hybrid  keyboard-game  controller  and 
found  that  it  needed  more  refinement  to  be  an  alternative  solution  for  Army  keyboard  and  control 
tasks.  The  dearth  of  publicly  available  research  is  believed  to  result  from  the  fact  that  any  such 
information  is  proprietary  to  highly  competitive  commercial  game  companies.  Nonetheless,  the 
lack  of  evidence  regarding  the  usefulness  of  game  controllers  is  not  a  sufficient  reason  to 
eliminate  the  game  controller  as  a  possible  control  device  for  control  systems  used  in  moving 
vehicles.  Game  controllers  and  game  pads  have  been  used  successfully  in  a  variety  of 
nonmilitary  environments,  including  OTM  automotive  applications  in  vehicles  such  as  minivans 
(Honda  Odyssey  Forum,  201 1).  Therefore,  a  video  console  game  controller  or  game  pad  may  be 
a  very  good  alternative  control  device  to  the  joystick  for  a  mobile  OTM  military  system. 

The  game  controller  has  become  a  commonly  used  electronic  device  familiar  to  many  potential 
users  (Console  Wars,  2006;  Game  Controller,  2006),  which  may  provide  positive  transfer  of 
training  to  military  control  systems.  Game  control  devices  may  provide  several  advantages  for 
use  in  moving  vehicle  environments.  To  be  effective,  conventional  control  devices  such  as  a 
mouse,  keyboard,  or  joystick  need  to  be  mounted  or  placed  on  a  surface,  such  as  a  desktop  for  a 
mouse.  One  advantage  of  the  game  controller  is  that  it  is  a  self-contained,  hand-held  device,  i.e., 
no  mounting  is  necessary  for  use.  Another  advantage  is  that  the  game  controller  is  portable  and 
can  easily  be  passed  from  one  Soldier  to  another,  without  regard  to  their  seating  position  or 
nearby  fixtures,  to  promote  interaction  and  easy  use  during  mobile  operations.  The  game 
controller  is  also  relatively  small  and  can  be  easily  stowed  in  a  carrier  when  not  in  use;  this 
means,  of  course,  that  it  also  can  be  more  easily  misplaced  than  a  mounted  controller. 

Operators  need  to  rely  on  stabilization  points  to  brace  their  hands,  wrists,  and  forearms  when 
performing  control  tasks  with  conventional  controls  in  moving  vehicles  or  erratic  movements 
occur.  Another  game  controller  advantage  is  that  it  is  easily  held  in  the  hand,  which  eliminates 
the  effects  of  large  skeletal  movements;  the  hand  is  the  reference  point  of  the  controller,  enabling 
the  thumbs  and  forefingers  to  manipulate  controller  functions.  This  should  help  minimize  erratic 
control  movements  caused  by  vehicle  motion  and  vibration. 
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2.  Research  Objective 


The  primary  objective  of  this  study  was  to  determine  the  usability  of  three  different  computer 
game  control  devices  (trackball,  joystick,  and  Xbox  360  Controller*)  for  Army  HRI  tasks 
performed  in  a  moving  Army  vehicle.  The  second  objective  was  to  determine  the  extent  to 
which  vehicle  operation  in  different  conditions  (baseline  engine  idle,  gravel  roads,  and  cross¬ 
country  terrain)  affects  the  usability  of  the  different  controllers. 

Several  hypotheses  were  developed.  The  first  was  that  vehicle  motion  would  negatively  impact 
the  ability  of  a  user  to  operate  all  controllers  for  HRI  mission  planning  and  teleoperation  tasks. 
The  second  hypothesis  was  that  the  Xbox  controller  would  provide  a  performance  advantage 
over  the  joystick  and  trackball  on  rough  terrain  because  the  hand-held  device  would  be  easier  to 
stabilize  against  the  body  and  enable  the  controls  to  be  referenced  to  the  operator’s  hand.  The 
latter  would  allow  more  efficient  use  of  the  operator’s  thumbs  and  fingers,  enabling  more 
efficient  control  manipulation.  The  third  hypothesis  was  that  terrain  types  with  a  greater 
variation  in  surface  would  have  a  more  significant  effect  on  controller  performance.  Therefore, 
cross-country  terrain  was  hypothesized  to  produce  the  greatest  Soldier  performance  decrements 
from  the  baseline  vehicle  idle. 


3.  Method 


3.1  Participants 

The  study  engaged  a  total  of  12  adults,  ranging  from  18  to  56  years  of  age.  Participants  were 
ARL  civilians  who  volunteered  during  normal  duty  hours  and  were  not  compensated  for  their 
role  in  this  study  beyond  their  normal  salary  or  wages.  The  participants  were  prescreened,  and 
those  selected  reported  a  low  susceptibility  to  motion  or  simulator  sickness,  no  medical 
conditions  that  might  be  affected  by  motion,  and  normal  visual  functioning  (including  corrected 
vision)  of  20/40  or  better. 

3.2  Apparatus 

The  apparatus  consisted  of  a  high-mobility,  multipurpose,  wheeled  vehicle  (HMMWV)  with  a 
mounted  shelter,  a  workstation  computer,  control  devices,  and  mission  planning  software. 


* 


Xbox  360  Controller  is  a  registered  trademark  of  Microsoft  Corporation. 
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3.2.1  HMMWV 


The  HMMWV  was  equipped  with  a  diesel  engine,  automatic  transmission,  and  four-wheel  drive. 
Figure  1  shows  the  HMMWV  vehicle  with  shelter  used  in  this  study. 


Figure  1 .  HMMWV  with  shelter. 


As  can  be  seen  in  figure  1,  a  hard- walled  shelter  was  mounted  on  the  rear  portion  of  the 
HMMWV.  The  shelter,  which  has  been  bolted  through  its  floor  to  structural  components  of  the 
HMMWV,  was  equipped  with  two  workstations  and  two  seats  (one  each  for  the  participant  and 
experimenter).  The  seats  were  pedestal-mounted,  forward-facing,  high-backed  seats  with 
adjustments  in  vertical,  lateral,  and  horizontal  dimensions  as  well  as  rearward  reclining,  and  were 
equipped  with  lap  belts.  For  this  study,  the  seats  were  adjusted  for  participant  comfort  and  to 
allow  a  viewing  distance  of  ~20  in  to  the  workstation  screen.  The  participant’s  workstation  had 
an  18-in  computer  monitor,  keyboard,  joystick,  X-Box  controller,  and  trackball  controller. 
Between  the  seats  and  the  monitors,  28  in  from  the  floor,  was  an  11-in-deep  keyboard  shelf  with 
a  10°  slope  that  runs  the  width  of  the  vehicle.  This  shelf  was  where  the  joystick  and  trackball 
were  mounted  for  the  experiment.  The  X-Box  controller  is  hand-held,  so  it  was  not  mounted. 

During  the  experiment,  the  participants  rode  in  the  left  seat.  The  experimenter  rode  on  the  seat 
next  to  the  participant,  and  thus  had  immediate  visual  and  physical  access  to  the  participant. 
Figure  2  shows  a  schematic  diagram  of  the  shelter  interior. 
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Figure  2.  Representation  of  the  interior  layout  of  the  HMMWV 
shelter.  Top  is  the  overhead  view  and  bottom  is  the 
driver-side  view. 

The  interior  of  the  vehicle  was  lit  by  ceiling-mounted  light  fixtures.  Ventilation  to  the  interior 
was  provided  by  a  roof-mounted  air  conditioner  and  heater  unit.  A  three-foot  step  ladder 
attached  to  the  shelter  was  used  for  shelter  ingress  and  egress.  For  emergency  egress,  there  was 
a  permanently  mounted  ladder  beside  the  outer  door.  Emergency  egress  procedures  were 
covered  in  the  participant  safety  training.  Movement  into  and  out  of  the  workstation  seats  was 
performed  via  the  16-in  (minimum)  passageway  between  the  seats  or  between  the  seats  and  the 
shelter  walls. 

Several  safety  measures  were  adopted.  The  experimenter  and  participant  were  able  to  maintain 
primary  communication  with  the  vehicle  driver  via  radio  in  case  of  emergency.  Secondary 
communication  was  via  a  button  mounted  on  the  wall  to  the  left  of  the  experimenter;  if  pressed,  a 
red  light  on  the  driver’s  console  was  activated  and  the  driver  would  immediately  come  to  a 
controlled  stop.  The  vehicle  driver  was  licensed  and  qualified  to  drive  the  HMMWV.  Before 
the  study,  the  vehicle  was  checked  by  experienced  vehicle  safety  personnel  from  the  Aberdeen 
Test  Center  Vehicle  Safety  Office,  who  inspected  the  vehicle  exterior  and  interior  to  ensure  that 
it  was  safe  for  participants  and  experimenters. 

During  this  study,  vehicle  ride  dynamics  were  measured  with  an  accelerometer  to  quantify 
vehicle  vibration  characteristics.  The  accelerometer  was  noninvasive  to  the  participant;  the 
accelerometer  was  embedded  within  a  rubber  pad  that  rested  upon  the  participant’s  seat  pad. 

Ride  dynamics  were  collected  to  compare  ride  characteristics  between  participants  as  needed,  as 
well  as  to  compare  ride  characteristics  between  past  and  future  experiments. 

3.2.2  Workstation  Computer 

The  participant’s  workstation  computer  was  a  ruggedized  Z  Microsystems  Intel-based  system 
with  an  Intel  motherboard,  2.2-GHz  P4  CPU,  512  MB  of  RAM,  and  an  ASUS  V9180  Magic 
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GeForce  4  MX440,  64-MB  video  card.  The  displays  were  Computer  Dynamics  rack-mounted, 
flat-panel,  touch-screen  displays  with  1280  x  1024  pixel  resolution.  These  displays  featured  an 
18-in,  active-matrix,  thin  film  transistor  liquid  crystal  display  with  170°  viewing  cone  and 
235-nit  (candela  per  square  meter)  brightness.  The  display  also  featured  350: 1  contrast  ratio  and 
25-ms  typical  pixel  response  time. 

2.2.3  Control  Devices 

The  three  controller  devices  used  in  this  experiment  were:  (1)  a  Microsoft  Xbox  360  Controller 
for  Windows  Game  Pad  (referred  to  as  Xbox),  (2)  a  CH  Products  IP  desktop  joystick  (with  three 
degrees  of  freedom)  (referred  to  as  joystick),  and  (3)  a  Microsoft  Trackball  Explorer  (referred  to 
as  trackball).  These  controllers  are  seen  in  figures  3-5.  The  Xbox  was  selected  as  the 
representative  game  pad  because  of  its  ability  to  interface  to  Windows.  The  joystick  was 
selected  because  it  most  closely  resembled  the  joystick  used  by  the  ARL  Robotics  Program 
Office.  The  trackball  was  chosen  because  of  its  successful  operation  in  other  ARL  Human 
Research  and  Engineering  Directorate  (HRED)  HMMWV  studies  (e.g.,  Haas  and  Stachowiak, 
2007). 


Figure  3.  Microsoft  Xbox  360  controller  for  Windows. 


Figure  4.  CH  products  IP  desktop  joystick. 
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Figure  5.  Microsoft  Trackball  Explorer. 


3.2.4  Mission  Planning  Software 

Custom  software  was  designed  to  present  cursor-control  HRI  mission  planning  tasks  and  to 
collect  time  and  error  data  associated  with  experimental  tasks.  Four  cursor-based  mission 
planning  tasks  (point-to-point  move  and  object  selection;  drag  and  drop;  select  and  hold;  and 
mouse  drag  area  selection)  were  chosen  because  they  incorporated  elements  of  basic  control 
functions  and  represented  common  unmanned  vehicle  mission  planning  tasks  associated  with  the 
Robotics  Collaborative  Technology  Alliance  (RCTA)  Tactical  Control  Unit  (TCU).  The 
software  was  designed  to  emulate  the  RCTA  TCU  in  appearance  and  present  representative 
robotics  usability  tasks,  while  collecting  time  and  accuracy  data  on  those  tasks. 


The  RCTA  TCU  is  a  map-based  control  and  display  system  for  Soldiers  to  plan  and  monitor 
unmanned  vehicle  missions.  Figure  6  shows  the  study’s  custom  software  screen,  which  is  based 
on  the  TCU. 


Figure  6.  Example  of  custom  software  screen. 


7 


3.3  Questionnaires 

Questionnaires  consisted  of  a  demographics  and  health  screening  questionnaire,  symptom 
questionnaire,  on-the-spot  workload  questionnaire,  and  evaluation  questionnaire.  The 
demographics  and  health  screening  questionnaire  was  given  at  the  beginning  of  each  test  day. 

The  symptom  questionnaire,  adapted  from  Gianaros  et  al.  (2001)  was  given  at  the  beginning  of 
each  day  and  after  each  condition  to  determine  the  participant’s  physical  motion  sickness 
symptoms  at  that  moment  in  time.  After  data  collection  trials,  the  participant  provided  an  on- 
the-spot  single  workload  rating  estimate  (Hart  and  Bortolissi,  1984;  Moray  et  al.,  2006)  in  which 
he  or  she  verbally  assigned  a  number  between  1  and  10  to  describe  the  workload  associated  with 
that  task,  with  1  representing  very  low  and  10  representing  very  high  workload  estimates.  The 
evaluation  questionnaire  was  given  after  each  movement  condition. 

3.4  Test  Course 

This  study  was  conducted  at  the  ARL  Ground  Vehicle  Experimental  Course  (GVEC)  at 
Aberdeen  Proving  Ground,  MD.  The  course  is  ~4  acres  in  size.  The  specific  vehicle  paths  were 
a  combination  of  secondary  (gravel  road)  and  off-road  (cross-country)  courses.  The  gravel  road 
was  flat  and  encircled  the  cross-country  course,  which  was  mostly  flat,  with  several  small  hills 
and  ridges  to  create  vehicle  pitch,  roll,  and  yaw  motion.  The  course  was  surrounded  by  “jersey 
barriers”  to  prevent  people  or  animals  from  straying  onto  the  course  and  prevent  the  HMMWV 
from  leaving  the  course.  Vehicle  speed  on  all  courses  was  controlled  by  a  qualified  HRED 
driver,  and  was  generally  between  10  and  15  mph.  A  trailer  on  the  course  served  as  the 
experiment  base,  data  collection  area,  and  participant  rest  area.  The  trailer  was  separated  from 
the  active  part  of  the  course  by  a  ditch  and  a  line  of  jersey  barriers.  Attached  to  the  trailer  was  an 
observation  deck. 

3.5  Ride  Quality  of  HMMWV  on  Test  Course 

Ride  quality  data  was  collected  using  a  seat  pad  accelerometer,  installed  in  the  participant’s  seat, 
and  an  Army  Test  Center  advanced  distributed  modular  acquisition  system  (ADMAS)  data 
collection  system.  Vehicle  vibration  crest  factor  (the  peak  amplitude  of  the  waveform  divided  by 
the  root  mean  square  amplitude  value)  was  derived  from  accelerometer  recordings  for  each 
movement  session  to  characterize  the  ride  quality.  Crest  factor  is  a  single  value  used  to  describe 
the  amplitude  of  vibration  in  each  movement  session. 

Table  1  contains  the  root  mean  square  amplitude  (RMSA)  and  crest  factor  for  each  terrain 
(ISO  2631-1,  1997).  The  RMS  As  and  crest  factors  indicate  that  participants’  ride  quality 
deteriorated  as  they  experienced  idle,  gravel,  and  cross-country  conditions,  in  that  order. 
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Table  1.  Ride  quality  as  described  by  root  mean  square  amplitude  (RMS  A)  and  crest  factor  (CF) 
for  seat  pad  vertical,  translational,  and  longitudinal  accelerations  (in  gs)  for  each  terrain. 


Idle 

Gravel 

Cross-Country 

RMSA 

CF 

RMSA 

CF 

RMSA 

CF 

Seat  pad  vertical 
acceleration 

0.026 

5.276 

0.142 

4.075 

0.191 

29.609 

Seat  pad  translational 
acceleration 

0.018 

3.101 

0.088 

4.529 

0.157 

30.717 

Seat  pad  longitudinal 
acceleration 

Combined  acceleration 

jx2  +  y2  +  z2 

0.043 

2.183 

0.072 

4.186 

0.156 

36.581 

0.053 

6.498 

0.182 

7.392 

0.292 

56.199 

3.6  Procedure 

The  order  of  events  in  this  study  is  described  in  table  2.  One  participant  was  tested  at  a  time. 
Participants  were  taken  to  the  GVEC  trailer,  where  they  were  given  a  volunteer  agreement 
affidavit  that  described  the  study  and  possible  risks.  They  were  informed  that  they  might 
withdraw  from  the  experiment  at  any  time  or  for  any  reason  without  reprisal.  Each  participant 
signed  the  consent  form  and  was  given  a  copy  of  the  signed  form,  which  included  points  of 
contact.  Participants  were  screened  for  self-reported  health  and  vision  conditions,  and 
susceptibility  to  simulator  and  motion  sickness.  Participants  who  met  the  experimental  criteria 
filled  out  the  volunteer  agreement  affidavit  and  the  demographics  and  health  screening  and 
symptom  questionnaires. 
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Table  2.  Order  of  events. 


Order 

Event 

Approximate  Time 
(min) 

1 

Consent  form 

5 

2 

Demographic  and  health  screening 
questionnaire; 

Symptom  questionnaire  (baseline) 

5 

3 

Task  training 

30 

4 

Break; 

Symptom  questionnaire 

15 

5 

Movement  condition  I 

Controller  1  task. 

On  the  spot  workload  evaluation; 
Controller  2  task. 

On  the  spot  workload  evaluation; 
Controller  3  task. 

On  the  spot  workload  evaluation 

30 

6 

Symptom  questionnaire; 

Evaluation  questionnaire 

10 

7 

Break; 

Symptom  questionnaire 

15 

8 

Movement  condition  II 

Controller  1  task. 

On  the  spot  workload  evaluation; 
Controller  2  task. 

On  the  spot  workload  evaluation; 
Controller  3  task. 

On  the  spot  workload  evaluation 

30 

9 

Symptom  questionnaire; 

Evaluation  questionnaire 

10 

10 

Break; 

Symptom  questionnaire 

15 

11 

Movement  condition  III 

Controller  1  task. 

On  the  spot  workload  evaluation; 
Controller  2  task. 

On  the  spot  workload  evaluation; 
Controller  3  task. 

On  the  spot  workload  evaluation 

30 

12 

Symptom  questionnaire; 

Evaluation  questionnaire 

10 

13 

Wait/recovery  period 

20 

Note:  Approximate  total:  3.75  h. 
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After  the  initial  questionnaires  were  filled  out,  the  participant  was  then  taken  to  the  HMMWV 
shelter  while  the  vehicle  engine  was  in  idle  and  received  training  on  the  study  set-up,  scenario, 
and  specific  tasks.  Each  participant  was  familiarized  with  the  vehicle,  particularly  safety  and 
emergency  egress  procedures.  The  participant  also  learned  how  to  provide  on-the-spot  single 
workload  rating  estimates.  After  this,  the  participant  learned  how  to  perform  the  experimental 
tasks  (point-to-point  move  and  selection;  select  and  hold;  drag  and  drop;  and  mouse-drag  area 
selection)  with  the  experimental  controller  for  that  condition.  Time  and  error  data  was 
automatically  collected  by  the  software.  A  description  of  these  tasks  follows: 

1.  Point-to-point  move  and  selection  of  objects  involved  using  the  control  device  to  select  a 
button  on  the  screen,  move  the  cursor  to  a  target  icon  on  the  screen,  select  the  icon,  move 
to  a  second  button,  and  select  it.  Time  and  accuracy  were  recorded.  This  task  emulated  a 
target  acquisition  task  in  which  targets  were  deleted.  The  “next”  button  was  selected 
(clicked  on)  to  start  the  task;  the  curser  was  moved  to  a  target  icon,  which  was  then 
selected;  the  curser  was  then  moved  to  the  “delete”  target  button  to  complete  the  task. 

Time  was  measured  from  the  click  of  the  “next”  button  until  the  click  of  the  “delete”  button 
at  the  end  of  each  selection.  Errors  included  clicking  the  background  or  wrong  buttons 
instead  of  the  target. 

2.  Select  and  hold  involved  moving  the  cursor  to  an  icon,  pushing  the  controller  button  to 
select  the  icon,  and  holding  the  button.  If  it  took  multiple  pushes  of  the  controller  button  to 
complete  the  task,  it  was  considered  an  error.  This  task  emulated  a  target  acquisition  task 
in  which  a  target  icon  was  moved  into  a  “target  zone”  by  selecting  a  directional  arrow 
button  and  held;  the  target  moved  in  the  direction  of  the  arrow  as  long  as  the  arrow  button 
was  depressed.  Each  target  movement  required  the  use  of  two  arrow  buttons;  hold  time 
was  measured  from  the  “mouse  down”  time  the  arrow  button  was  clicked  until  the  “mouse 
up”  time  the  arrow  button  was  released.  Errors  included  multiple  clicks  on  an  arrow  button 
instead  of  a  hold  and  click  elsewhere  on  the  screen. 

3.  Drag  and  drop  involved  moving  the  cursor  to  an  icon,  pushing  a  button  to  select  the  icon, 
and  then  holding  the  button  while  dragging  the  icon  to  a  predetermined  position  on  the 
screen.  Upon  reaching  the  predetermined  position,  the  select  button  was  released  to  drop 
the  icon.  This  task  emulated  a  target  acquisition  task  in  which  a  target  icon  was  moved  into 
a  “target  zone”  by  selecting  and  dragging  the  target  icon  to  the  “target  zone.”  Time  was 
measured  from  the  “mouse  down”  time  the  icon  was  clicked  until  the  “mouse  up”  time 
when  the  icon  was  dropped  in  the  “target  zone”  by  releasing  the  button.  Errors  included 
clicking  the  background  instead  of  the  target  and  when  the  icon  was  dropped  outside  of  the 
target  zone. 
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4.  Mouse-drag  area  selection  involved  moving  the  cursor  to  a  position  on  the  screen  in  a  text 
box,  pushing  a  button  to  put  the  cursor  into  “select  mode,”  then  dragging  the  cursor  to 
select  all  of  the  text  in  the  text  box  except  for  the  first  character.  This  task  emulated  a 
mission  planning  text  selection  task,  and  required  a  great  deal  of  dexterity  to  position  the 
cursor  between  letters  of  text.  The  “next”  button  was  selected  (clicked  on)  to  start  the 
task;  the  text  was  then  selected  in  a  text  box  by  the  click,  drag,  and  release  method;  then  a 
clear  button  was  selected  to  end  the  task.  Time  was  measured  from  the  click  of  the  “next” 
button  until  the  click  of  the  “clear”  button  at  the  end  of  each  selection.  Errors  included 
selection  of  text  instead  of  the  indicated  area. 

The  participants  executed  the  controller  tasks  in  a  software  training  mode.  The  training 
consisted  of  20  repetitions  of  each  task  with  each  controller.  After  each  training  task  with  each 
controller,  the  participant  was  asked  if  they  would  like  to  receive  more  training,  and  were 
permitted  more  training  until  they  felt  competent  in  the  performance  of  that  task. 

After  training,  experimental  conditions  were  presented  to  each  participant.  Each  task  was 
presented  10  times  for  each  controller  within  one  movement  condition  (engine  idle,  gravel  road, 
or  cross-country  terrain).  Each  movement  condition  lasted  for  ~30  min.  Since  three  controllers 
were  used  in  each  movement  condition,  the  participant  performed  a  total  of  120  experimental 
trials  (10  sequential  repetitions  x  4  tasks  x  3  controllers)  in  each  movement  condition.  The 
participant  had  a  short  break  (15-30  min)  at  the  GVEC  trailer  between  each  movement 
condition. 

After  each  movement  condition,  the  participant  was  asked  if  he  or  she  would  like  to  continue 
participation  in  the  remaining  conditions  (if  there  were  any).  If  they  agreed  to  continue,  the  trials 
continued  as  planned,  beginning  with  the  next  experimental  condition.  If  any  participant  did  not 
wish  to  continue,  the  trials  were  abandoned  and  the  experiment  terminated  for  that  subject.  If  the 
participant  agreed  to  continue,  he  or  she  was  specifically  reminded  that  they  could  end  the  trial  at 
any  time.  If  the  participant  experienced  any  form  of  sickness  during  or  after  the  experiment,  he 
or  she  was  permitted  to  recuperate  under  the  supervision  of  study  personnel. 

As  can  be  seen  in  table  1,  before  each  movement  condition,  the  participant  filled  out  a  symptom 
questionnaire  Between  blocks  of  controller  tasks,  the  also  participant  provided  an  on-the-spot 
single  workload  rating  estimate.  After  each  movement  condition,  the  participant  completed  a 
symptom  questionnaire  and  an  evaluation  questionnaire. 

After  completing  the  last  task  in  the  last  movement  condition,  the  experiment  ended  and  the 
participant  was  brought  to  the  GVEC  trailer,  where  he  or  she  completed  the  final  evaluation 
questionnaire  and  symptom  questionnaire.  The  participant  was  then  told  to  remain  at  the  test  site 
for  at  least  30  min  after  the  last  trial,  or  until  they  had  minimal  symptoms,  and  was  told  not  to 
drive  until  he  or  she  had  been  free  of  dizziness  or  vertigo  for  at  least  1  h.  Just  prior  to  dismissal 
from  the  experimental  site,  the  participants  were  asked  to  complete  a  final  symptom 
questionnaire  to  document  their  symptom  state. 
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Time  and  error  data  in  each  condition  were  automatically  collected  by  the  data  measurement 
software. 

3.7  Experimental  Design 

The  treatment  structure  for  all  controller  tasks  was  a  3  x  3  within-subjects  factorial  design. 

3.7.1  Independent  Variables 

The  independent  variables  were  as  follows: 

1.  Controller  type  was  a  within-participant  variable.  The  three  categories  of  controller  were: 

a.  Microsoft  Xbox  360  Controller  for  Windows  Game  Pad 

b.  CH  Products  IP  desktop  joystick 

c.  Microsoft  Trackball  Explorer 

2.  Movement  Condition  was  a  within-participant  variable.  The  three  categories  of  movement 
condition  were: 

a.  Vehicle  at  a  stop,  engine  idling 

b.  Vehicle  traveling  over  cross-country  terrain  at  8-12  mph 

c.  Vehicle  traveling  over  gravel  road  at  10-15  mph 

Movement  conditions  were  assigned  to  participants  by  means  of  a  double  reverse,  Latin  Square 
design.  The  Latin  Square  was  counterbalanced  with  respect  to  order  of  the  treatments  to  ensure 
that  each  treatment  was  administered  first,  second,  and  last,  and  no  treatment  consistently 
followed  another.  Part  way  through  the  experiment,  movement  order  was  changed  so  that  the 
cross-country  movement  condition  would  occur  last  for  each  subject,  with  the  first  two 
conditions  counterbalanced  with  the  original  Latin  Square.  This  change  occurred  because  a  large 
proportion  of  participants  became  ill  during  the  cross-country  condition  and  dropped  out  of  the 
experiment.  This  change  in  presentation  order  was  made  to  retain  as  many  participants  as  possible. 

The  order  in  which  each  controller  was  assigned  to  each  of  the  three  movement  conditions  for 
each  participant  was  counterbalanced  with  three  double  reverse,  Latin  Squares  to  ensure  that 
each  controller  was  administered  first,  second,  and  last,  and  that  no  controller  consistently 
followed  another. 

3.7.2  Dependent  Variables 

The  four  experimental  tasks  performed  during  each  condition  were  click  and  hold,  drag  and  drop, 
point-to-point,  and  text  select.  Lor  the  click  and  hold  task,  the  elapsed  time  and  number  of 
controller  clicks  to  complete  the  task  were  obtained.  Data  for  the  drag  and  drop  task  were 
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the  elapsed  time  to  complete  each  task,  number  of  clicks,  and  number  of  missed  drags.  The 
point-to-point  task  data  were  time  to  select  the  target,  number  of  errors,  and  time  to  delete  the 
target.  For  the  text  select  task,  data  were  time  to  select  the  target  text  and  number  of  characters 
selected  within  the  target  text. 

3.8  Data  Analysis 

Where  response  time  data  were  positively  skewed,  a  natural  log  transformation  was  performed  to 
give  the  data  a  normal  distribution  (Cleveland,  1984).  For  all  dependent  variables,  data  were 
analyzed  using  separate  analyses  of  variance  (ANOVA).  The  ANOVAs  were  run  using  an  SPSS 
Linear  Mixed  Model  Analysis. 

As  previously  noted,  because  many  participants  became  ill  after  cross-country  terrain,  the  order 
of  presentation  of  terrain  was  changed  part  way  through  the  study  to  permit  the  retention  of 
participants.  Thus,  cross-country  terrain  was  presented  as  the  last  terrain  condition  for  10  out  of 
12  participants.  Because  the  order  of  terrain  presentation  was  not  counterbalanced,  a  statistical 
analysis  was  performed  to  take  this  into  account.  In  addition  to  the  controller  and  terrain 
variables,  two  order  variables  (order  of  presentation  of  terrain  and  order  of  presentation  of 
controller)  and  a  variable  for  sequence  of  terrain  presentation  (terrain  sequence)  were  used  in  the 
statistical  analysis  to  account  for  order  of  terrain  presentation.  The  SPSS  Linear  Mixed  Model 
procedure  was  run  using  terrain,  terrain  order,  terrain  sequence,  controller,  controller  order,  and 
terrain  x  controller  as  fixed  effects.  Random  effects  used  were  sequence  within  subject  and 
terrain  x  terrain  order  x  sequence  within  subject. 

Interactions  between  the  random  and  fixed  effect  factors  were  used  as  error  terms  for  testing 
hypotheses  about  the  fixed  effect  factors.  Effects  showing  a  probability  value  p  greater  than  0.05 
were  not  considered  statistically  significant.  The  least  significant  difference  post  hoc  test  was 
performed  at  p  <  0.05,  but  only  if  the  corresponding  test  is  statistically  significant  at  the  0.05 
level. 


4.  Results 


4.1  Click  and  Hold 

4.1.1  Elapsed  Time 

The  SPSS  mixed  model  ANOVA  performed  on  the  log-transformed  data  indicated  significant 
main  effects  for  terrain  (F  =  17.59,  p  =  0.00)  and  controller  (F  =  3.02;  p  =  0.05).  There  were  no 
other  significant  main  effects  or  interactions.  Post  hoc  testing  indicated  that  there  was  a 
significant  difference  between  each  terrain,  with  cross-country  terrain  having  the  highest  elapsed 
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time  (11.58  ±  5.6  s  mean  ±  s.d.  reported  throughout),  gravel  having  a  significantly  lower  elapsed 
time  (9.69  ±  4.92  s),  and  idle  engine  having  the  lowest  elapsed  time  (9.47  ±  4.10  s)  for  the  click 
and  hold  task. 

The  post  hoc  test  on  the  transformed  controller  data  indicated  that  mean  elapsed  time  was 
shortest  for  the  trackball  (9.70  +  4.11  s),  with  no  significant  difference  between  the  Xbox 
(10.52  ±  5.25  s)  and  the  joystick  (10.38  ±  5.43  s). 

4.1.2  Number  of  Clicks 

The  ANOVA  indicated  significant  main  effects  for  controller  (F  =  9.41;  p  =  0.00),  with  no  other 
significant  main  effects  or  interactions.  The  post  hoc  test  indicated  that  the  mean  number  of 
clicks  required  for  the  joystick  (2.65  ±  1.25)  was  significantly  lower  than  that  required  by  the 
Xbox  (2.87  ±  1.25)  and  trackball  (3.08  ±  1.83)  controllers.  There  was  no  significant  difference 
between  the  trackball  and  Xbox  controllers. 

4.2  Drag  and  Drop 
4.2.1  Elapsed  Time 

The  ANOVA  performed  on  the  transformed  data  indicated  a  significant  interaction  effects  for 
terrain  x  controller  (F  =  6.27;  p  =  0.00),  and  significant  main  effects  for  terrain  (F  =  25.96; 
p  =  0.00)  and  for  controller  (F  =  28.11;  p  =  0.00). 

A  post  hoc  test  on  the  terrain  x  controller  data  (shown  in  figure  7)  indicated  that  on  idle  and 
gravel  terrain,  there  were  significant  differences  between  all  controllers.  The  trackball  provided 
a  significantly  shorter  response  time,  and  the  Xbox  registered  the  greatest  response  time  on  idle 
and  gravel  terrain.  There  was  no  significant  difference  between  controllers  on  cross-country 
terrain. 

A  post  hoc  test  on  the  terrain  data  indicated  that  elapsed  time  was  significantly  greater  on  cross¬ 
country  terrain  (7.99  +  4.48  s)  than  on  gravel  (5.74  +  4.48  s)  or  idle  (5.78  ±  4.48  s)  terrain.  There 
was  no  significant  difference  between  gravel  and  idle  terrain. 

As  can  be  seen  in  figure  7,  an  analysis  of  controller  main  effect  data  is  precluded  by  the  data;  the 
relationship  between  controllers  for  cross-country  conditions  is  different  from  those  for  idle  and 
gravel  conditions. 
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Figure  7.  Mean  drag-and-drop  elapsed  response  time  (seconds)  for  the  terrain  x  controller 
interaction. 

4.2.2  Number  of  Background  Clicks 

The  number  of  background  clicks  is  an  error  measure,  signifying  the  number  of  incorrect  clicks 
made  on  nontarget  objects.  The  data  indicated  that  out  of  1 166  trials,  198  (17%)  were 
characterized  as  background  clicks.  Because  this  is  a  small  sample  size,  a  nonparametric 
Kruskal-Wallis  test  for  significant  differences  between  means  was  conducted  to  determine 
whether  terrain  and/or  controller  had  a  significant  effect  on  the  number  of  background  clicks. 

For  terrain  type,  there  were  29  background  clicks  at  engine  idle,  71  on  gravel  terrain,  and  98  on 
cross-country.  However,  the  Kruskal-Wallis  test  indicated  that  there  was  no  significant 
difference  (p  >  0.05)  for  terrain  type. 

For  controller  type,  the  Xbox  had  35  background  clicks,  the  joystick  had  44,  and  the  trackball 
had  1 19.  The  Kruskal-Wallis  test  indicated  that  there  was  a  significant  difference  (p  =  0.029)  for 
controller  type.  The  post  hoc  median  test  indicated  that  the  trackball  had  a  significantly  greater 
number  of  background  clicks  than  the  other  two  controllers,  and  that  there  were  no  other 
significant  differences. 
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4.2.3  Number  of  Missed  Drags 

The  number  of  missed  drags  is  an  error  measure  signifying  the  number  of  drags  missed.  The 
data  indicated  that  out  of  1 166  trials,  128  (11%)  were  characterized  as  background  clicks 
indicative  of  a  missed  drag. 

For  terrain  type,  there  were  26  background  clicks  at  engine  idle,  52  on  gravel  terrain,  and  50  on 
cross-country.  However,  the  Kruskal-Wallis  test  indicated  that  there  was  no  significant 
difference  (p  >  0.05)  between  means  for  terrain  type. 

For  controller  type,  the  Xbox  had  16  background  clicks,  the  joystick  had  46,  and  the  trackball 
had  66.  The  Kruskal-Wallis  test  indicated  that  there  was  a  significant  difference  (p  =  0.032)  for 
controller  type.  The  post  hoc  median  test  indicated  that  the  joystick  and  trackball  had  a 
significantly  larger  number  of  background  clicks  than  the  Xbox.  There  were  no  other  significant 
differences. 

4.3  Point-To-Point 

4.3.1  Mean  Response  Time  to  Select 

The  ANOVA  performed  on  the  mean  time  to  select  point-to-point  targets  data  indicated 
significant  interaction  effects  for  terrain  x  controller  (F  =  9.18;  p  =  0.00)  and  significant  main 
effects  for  terrain  (F  =  19.03;  p  =  0.00)  and  controller  (F  =  32.85;  p  =  0.00).  Results  from  the 
post  hoc  test  for  the  terrain  x  controller  interaction  are  shown  in  figure  8.  Data  indicated  that  on 
idle  terrain,  all  means  were  significantly  different;  the  trackball  provided  significantly  lower 
mean  time  to  select  and  Xbox  the  highest.  On  gravel  terrain,  the  trackball  again  provided 
significantly  lower  mean  time  to  select,  while  the  joystick  and  Xbox  were  significantly  greater, 
but  not  significantly  different  from  each  other.  On  cross-country  terrain,  there  was  no  significant 
difference  between  controllers,  although  the  mean  response  time  to  select  was  significantly 
greater  for  all  controllers  than  any  other  condition. 

Figure  9  shows  data  for  the  terrain  main  effect.  The  mean  response  time  to  select  was 
significantly  greater  for  cross-country  terrain  than  either  idle  or  gravel.  There  was  no  significant 
difference  between  idle  or  gravel  terrains.  For  controller  type,  the  nature  of  the  interaction 
(changes  in  the  relationship  between  Xbox  and  joystick)  across  terrains  precluded  interpretation 
of  controller  main  effects. 
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Figure  8.  Mean  response  time  (seconds)  to  select  for  the  terrain  x  controller  interaction,  point-to-point  time  to 
select. 
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Figure  9.  Mean  response  time  to  select  (seconds)  for  terrain,  point-to-point  time  to  select. 


4.3.2  Mean  Response  Time  to  Delete 

The  ANOVA  performed  on  the  transformed  data  indicated  significant  interaction  effects  for 
terrain  x  controller  (F  =  5.64;  p  =  0.00),  and  significant  main  effects  for  terrain  (F  =  27.96; 
p  =  0.00)  and  controller  (F  =  41.61;  p  =  0.00).  The  means  for  the  terrain  x  controller  data  are 
shown  in  figure  10.  Post  hoc  results  indicated  that  the  trackball  provided  a  significantly  shorter 
mean  time  to  delete  across  all  terrains.  For  the  gravel  and  cross-country  terrains,  there  was  no 
significant  difference  between  the  joystick  and  the  Xbox,  though  each  was  greater  than  the 
trackball.  For  the  idle  condition,  the  joystick  had  a  significantly  shorter  mean  time  to  delete  than 
the  Xbox.  The  mean  response  time  to  delete  ranged  from  ~4.5  to  10.0  s.  Whether  these 
differences  could  make  a  practical  difference  in  U.S.  Army  mobile  operations  would  depend 
upon  the  particular  operation  and  mission  being  performed.  All  controllers  had  a  significantly 
greater  mean  response  time  to  delete  on  cross-country  terrain,  while  there  was  no  significant 
difference  between  idle  and  gravel  terrain. 
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Figure  10.  Mean  response  time  (seconds)  to  delete  for  the  terrain  x  controller  interaction. 


Figure  1 1  shows  data  for  the  terrain  main  effect.  The  mean  response  time  to  delete  was 
significantly  greater  for  cross-country  terrain  than  either  idle  or  gravel.  There  was  no  significant 
difference  between  idle  or  gravel  terrains.  For  controller  type,  the  nature  of  the  interaction 
(changes  in  the  relationship  between  Xbox  and  joystick)  precluded  interpretation  of  controller 
main  effects. 
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Figure  11.  Mean  response  time  (seconds)  to  delete  for  the  terrain. 


4.3.3  Mean  Number  of  Background  Clicks 

The  number  of  background  clicks  is  an  error  measure,  signifying  the  number  of  incorrect  clicks 
made  on  nontarget  objects.  The  mean  number  of  background  clicks  data  indicated  that  out  of 
1166  total  clicks,  169  (14%)  were  characterized  as  background  clicks. 

For  terrain,  35  background  clicks  were  performed  at  engine  idle,  39  on  gravel,  and  95  on  cross¬ 
country.  The  Kruskal- Wallis  test  indicated  that  there  was  there  was  a  significant  difference 
( p  =  0.033)  for  terrain  type.  The  median  test  indicated  that  the  number  of  background  clicks  on 
cross-country  terrain  was  significantly  higher  than  those  at  engine  idle  and  on  gravel  terrain. 

For  controller,  66  were  performed  with  the  Xbox,  62  with  the  trackball,  and  41  with  the  joystick. 
The  Kruskal-Wallis  test  indicated  that  there  was  there  was  no  significant  difference  (p  >  0.05)  for 
controller. 
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4.4  Text  Select 


4.4.1  Time  to  Select 

The  ANOVA  performed  on  the  transformed  data  indicated  significant  main  effects  for  controller 
(F  =  32.62;  p  =  0.00)  and  terrain  (F  =  1 1.39;  p  =  0.00),  with  no  significant  interaction. 

The  controller  data,  as  shown  in  figure  12,  indicated  that  the  mean  time  to  select  text  was 
significantly  shorter  for  the  trackball  than  for  the  Xbox  and  the  joystick,  with  no  significant 
difference  between  Xbox  and  joystick. 


Figure  12.  Mean  response  time  to  select  for  controllers. 
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The  post  hoc  data  for  terrain,  shown  in  figure  13,  indicated  that  mean  time  to  select  was  greatest 
for  cross-country  terrain,  shorter  for  gravel  terrain,  and  shortest  for  engine  idle  conditions.  All 
means  were  significantly  different. 


Figure  13.  Mean  response  time  to  select  for  terrain. 


4.4.2  Number  of  Text  Select  Errors 

The  error  data  indicated  that  out  of  1060  total  text  select  clicks,  185  (17%)  were  characterized  as 
errors.  For  terrain,  37  text  select  errors  were  made  on  gravel  terrain,  46  at  engine  idle,  and  102 
on  cross-country  terrain.  The  Kruskal- Wallis  test  indicated  that  there  was  there  was  no 
significant  difference  (p  >0.05)  for  type  of  terrain. 

For  controller,  51  errors  were  made  with  the  Xbox,  61  with  the  joystick,  and  73  with  the 
trackball.  The  Kruskal-Wallis  test  indicated  that  there  was  there  was  no  significant  difference 
{p  >  0.05)  for  type  of  controller. 
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4.5  Workload 

The  ANOVA  performed  on  the  workload  data  indicated  significant  main  effects  for  terrain 
(F  =  160.94;  p  =  0.00).  There  were  no  other  significant  main  effects  or  interactions.  The  mean 
workload  rating  (higher  numbers  indicate  greater  workload,  with  10  being  the  upper  limit) 
indicated  that  perceived  workload  was  greater  for  cross-country  terrain  (6.31)  than  for  gravel 
(4.35)  or  idle  (3.33)  conditions.  Each  mean  was  significantly  different.  This  indicated  that 
participants  regarded  workload  as  being  greatest  on  cross-country  terrain  and  least  at  engine  idle. 
Unfortunately,  the  data  collected  on  motion  sickness  was  inadvertently  destroyed. 


5.  Discussion 


The  primary  objective  of  this  study  was  to  determine  the  usability  of  three  different  computer 
game  control  devices  (trackball,  joystick,  and  Xbox  controller)  for  Army  HRI  tasks  performed  in 
a  moving  Army  vehicle.  The  second  objective  was  to  determine  the  extent  to  which  vehicle 
operation  in  different  conditions  (baseline  engine  idle,  gravel  roads,  and  cross-country  terrain) 
affects  the  usability  of  the  different  controllers. 

5.1  Controller 

The  hypothesis  stated  that  the  Xbox  controller  would  provide  a  performance  advantage  over  the 
joystick  and  trackball  on  cross-country  terrain  because  the  hand-held  Xbox  would  enable  more 
efficient  controller  manipulation.  Time  data  indicate  that  this  hypothesis  was  not  supported.  For 
the  few  tasks  that  did  have  a  significant  controller  main  effect,  different  controllers  were  found  to 
provide  the  best  performance. 

Analysis  of  click  and  hold  timing  measures  showed  a  significant  main  effect  for  controller,  with 
the  shortest  time  for  trackball  and  no  significant  difference  between  the  Xbox  and  joystick.  For 
click  and  hold  number  of  clicks,  the  joystick  provided  the  best  performance  (a  smaller  mean 
number  of  clicks),  with  no  significant  difference  between  the  Xbox  and  trackball.  For  text  select 
time -based  measure  (time  to  select  text),  the  trackball  showed  a  significantly  shorter  response 
time  than  for  the  Xbox  and  joystick,  with  no  significant  difference  between  the  latter  controllers. 

For  error-based  measures,  the  Xbox  provided  an  advantage  in  performance.  For  drag  and  drop 
number  of  background  clicks  and  number  of  missed  drags,  the  Xbox  had  the  smallest  number  of 
errors  and  the  trackball  had  the  greatest  number  of  errors.  However,  for  the  point-to-point  task, 
the  greatest  number  of  errors  was  logged  with  the  Xbox  and  the  trackball,  and  the  least  number 
with  the  joystick. 
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Together  these  results  indicate  that  the  Xbox  controller,  while  not  providing  the  best  time  data, 
did  have  an  advantage  in  error-based  measures  on  tasks  that  involved  holding  a  selection  button 
while  at  the  same  time  moving  the  cursor. 

5.2  Terrain 

The  hypothesis  was  that  terrain  types  with  greater  surface  variation  would  have  a  more 
significant  effect  on  controller  performance  in  Army-based  HRI  tasks.  As  a  result,  it  was 
hypothesized  that  cross-country  terrain  (0.292  total  RMSA  acceleration)  would  produce  the 
greatest  decrement  in  Soldier  performance,  followed  by  gravel  (0.182  total  RMSA  acceleration) 
and  then  idle  (0.053  total  RMSA  acceleration).  The  data  for  all  tasks  support  the  hypothesis 
regarding  cross-country  terrain,  and  all  tasks  showed  the  greatest  decrement  in  cross-country 
terrain.  However,  differences  between  gravel  and  engine  idle  conditions  depended  on  the 
particular  task. 

Analysis  of  click  and  hold,  drag  and  drop,  point-to-point,  and  text  select  timing  measures  (e.g., 
mean  elapsed  time  to  perform  the  task  for  click  and  hold  and  drag  and  drop;  time  to  select  for 
point-to-point  and  text  select;  and  mean  time  to  delete  for  point-to-point)  indicated  that  cross¬ 
country  terrain  required  the  greatest  time  to  perform  the  task  measures.  Some  timing  measures 
(click  and  hold,  text  select),  indicated  that  gravel  terrain  resulted  in  a  significantly  greater  time  to 
perform  the  task  than  engine  idle  conditions,  while  other  tasks  (drag  and  drop,  point-to-point) 
showed  no  significant  difference  between  gravel  and  idle  terrains. 

For  error  measures  such  as  number  of  background  clicks,  there  was  a  significantly  greater 
number  of  errors  for  controller  type  (trackball)  for  drag  and  drop,  while  terrain  (cross-country) 
had  a  significantly  greater  number  of  errors  for  point-to-point.  For  drag  and  drop  missed  drags, 
the  joystick  and  trackball  controllers  had  a  significantly  greater  number  of  errors  than  the  Xbox. 
There  were  no  other  significant  effects  in  any  of  the  error  tests. 

Workload  data  also  provided  information  regarding  terrain  effects.  Participants  indicated  greater 
perceived  workload  ratings  for  cross-country  terrain  than  for  gravel  or  idle  conditions,  with  each 
condition  significantly  different. 

Together  these  results  indicate  that  as  ride  quality  deteriorated  with  greater  terrain  surface 
variation,  the  operator’s  performance  using  all  of  the  controllers  declined. 

5.3  Interaction  Between  Terrain  and  Controller 

As  previously  noted,  there  were  several  significant  interaction  effects  between  terrain  and 
controller  for  different  tasks.  For  time-based  drag  and  drop  (mean  elapsed  time)  and  point-to- 
point  (mean  response  time  to  select)  tasks,  the  trackball  provided  a  significantly  shorter  response 
time  on  idle  and  gravel  terrain,  and  there  was  no  significant  difference  between  controllers  on 
cross-country  terrain.  For  drag  and  drop,  the  Xbox  provided  the  longest  response  time  on  idle 
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and  gravel  terrain.  For  point-to-point  mean  time  to  select,  the  Xbox  provided  a  significantly 
longer  mean  response  time  on  idle  terrain.  On  gravel  terrain,  the  joystick  and  Xbox  had  the 
longest  response  times,  but  were  not  significantly  different  from  each  other.  For  point-to-point 
mean  time  to  delete  data,  the  trackball  provided  a  significantly  shorter  time  to  delete  across  all 
terrains.  For  gravel  and  cross  country  terrains,  there  was  no  significant  difference  between  the 
joystick  and  the  Xbox.  For  the  idle  condition,  the  joystick  had  a  significantly  shorter  mean  time 
to  delete  than  the  Xbox. 

5.4  Issues  of  Interest 

One  may  question  why  the  Xbox  controller  did  not  show  a  significant  difference  in  error  and 
time  performance  over  the  other  two  controllers  on  cross-country  terrain.  The  Xbox  was 
hypothesized  to  have  a  performance  advantage  over  the  joystick  and  trackball  in  a  moving 
environment  because  it  is  held  in  the  hands,  which  was  thought  to  provide  more  of  a  reference 
point  to  enable  users’  thumbs  and  fingers  to  manipulate  the  controls  more  efficiently  than  with 
the  other  controllers.  However,  experimenter  observations  during  the  study  indicated  that 
participants  used  different  strategies  to  attempt  to  stabilize  each  of  the  controller  types.  Since  the 
participants  were  not  restricted  in  their  method  of  controller  use,  they  developed  different 
mechanisms  to  enable  moving  small  effectors  instead  of  large  limbs  to  increase  the  functionality 
of  each  device  in  the  moving  environments.  Participants  were  observed  holding  the  Xbox  close 
to  their  body  with  their  hands,  especially  during  trials  on  cross-country  terrain.  Users  were 
observed  attempting  to  stabilize  the  joystick  by  positioning  it  on  the  upper  leg  or  between  their 
legs.  Users  attempted  to  stabilize  the  trackball  by  holding  the  unit  to  the  surface  of  the  shelter 
with  one  hand,  and  using  the  dominant  hand  to  move  the  trackball.  In  general,  we  argue  that 
because  users  were  able  to  find  a  way  to  stabilize  each  type  of  controller,  the  Xbox  did  not  have 
a  performance  advantage. 

Another  issue  deals  with  the  relevancy  of  the  difference  in  time  for  terrains.  In  many  instances, 
although  some  differences  were  significant,  they  were  relatively  small  (i.e.,  differences  between 
the  highest  and  lowest  response  time  means  across  terrains  was  2.1 1  s  for  click  and  hold,  and 
2.25  s  for  drag  and  drop).  Whether  these  differences  would  have  an  effect  on  U.S.  Army  mobile 
operations  would  depend  upon  the  particular  operation  and  mission  being  performed. 

5.5  Future  Research 

Although  this  research  shows  that  the  gaming  console  may  have  some  advantages  over 
traditional  computer  input  devices  (joystick  and  trackball),  additional  work  is  needed  to 
understand  comparative  advantages  and  disadvantages  of  various  control  devices  for  various 
tasks  performed  within  moving  vehicles.  Future  research  could  also  explore  the  use  of  the  yoke 
and  touchscreen  as  control  devices.  Future  research  should  target  a  population  with  known 
gaming  experience  and  experience  with  handheld  controllers.  This  would  be  more  relevant  to 
the  population  of  the  young  Soldiers  that  would  be  the  projected  users  of  such  systems. 
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FORT  BELVOIR  VA  22060-5828 

1  ARMY  RSCH  LABORATORY  -  HRED 

RDRL  HRM  AY  M  BARNES 
2520  HEALY  AVE 
STE  1172  BLDG  51005 
FORT  HUACHUCA  AZ  85613-7069 

1  ARMY  RSCH  LABORATORY  -  HRED 

RDRL  HR  MP  D  UNGV ARSKY 
POPE  HALL  BLDG  470 
BCBL  806  HARRISON  DR 
FORT  LEAVENWORTH  KS  66027-2302 


1  ARMY  RSCH  LABORATORY  -  HRED 
RDRL  HRM  DQ  MR  FLETCHER 
NATICK  SOLDIER  CTR 
AMSRD  NSC  WS  E  BLDG  3  RM  343 
NATICK  MA  01760-5020 

1  ARMY  RSCH  LABORATORY  -  HRED 
RDRL  HRM  AT  J  CHEN 
12350  RESEARCH  PKWY 
ORLANDO  FL  32826-3276 

1  ARMY  RSCH  LABORATORY  -  HRED 
RDRL  HRM  AT  C  KORTENHAUS 
12350  RESEARCH  PKWY 
ORLANDO  FL  32826 

1  ARMY  RSCH  LABORATORY  -  HRED 
RDRL  HRM  AS  C  MANASCO 
SIGNAL  TOWERS 
BLDG  29808A  RM  303A 
FORT  GORDON  GA  30905-5233 

1  ARMY  RSCH  LABORATORY  -  HRED 
RDRL  HRM  CU 
6501  E  11  MILE  RD  MS  284 
BLDG200A  2ND  FL  RM2104 
WARREN  MI  48397-5000 

1  ARMY  RSCH  LABORATORY  -  HRED 
FIRES  CTR  OF  EXCELLENCE 
FIELD  ELEMENT 
RDRL  HRM  AF  C  HERNANDEZ 
3040  NW  AUSTIN  RD  RM  221 
FORT  SILL  OK  73503-9043 

1  ARMY  RSCH  LABORATORY  -  HRED 
RDRL  HRM  AV  S  MIDDLEBROOKS 
91012  STATION  AVE 
FORT  HOOD  TX  76544-5073 

1  ARMY  RSCH  LABORATORY  -  HRED 
HUMAN  RSRCH  AND  ENGRNG 
DIRCTRT  MCOE  FIELD  ELEMENT 
RDRL  HRM  DW  E  REDDEN 
6450  WAY  ST 
BLDG  2839  RM  310 
FORT  BENNING  GA  31905-5400 
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NO.  OF 

COPIES  ORGANIZATION 

1  ARMY  G1 
(CD  DAPE  MR 
only)  B  KNAPP 

300  ARMY  PENTAGON  RM  2C489 
WASHINGTON  DC  20310-0300 


ABERDEEN  PROVING  GROUND 

5  DIR  USARL 
RDRL  HR 
L  ALLENDER 
T  LETOWSKI 
RDRL  HRM 

P  S  A V AGE-KNEPS  HIELD 
RDRL  HRM  A 
D  HARRAH 
RDRL  HRS  D 
B  AMREIN 
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